Multiple-UAV Reinforcement Learning Algorithm Based on Improved PPO in Ray Framework

Distributed multi-agent collaborative decision-making technology is the key to general artificial intelligence. This paper takes self-developed Unity3D combat environment as test scenario, setting a task that requires heterogeneous unmanned aerial vehicles (UAVs) perform distributed and complete cooperation task. Aiming at problem of traditional proximal policy optimization (PPO) algorithm’s poor performance in field complex collaboration scenarios based on training framework Ray, Critic network PPO algorithm improved learn centralized value function, muti-agent (MAPPO) proposed. At same time, inheritance method course learning adopted improve generalization algorithm. In experiment, MAPPO can obtain highest average accumulate reward compared with other algorithms goal fewest steps after convergence, which fully demonstrates outperforms state-of-the-art.